Zero-Shot Audio Classification Via Semantic Embeddings

In this paper, we study zero-shot learning in audio classification via semantic embeddings extracted from textual labels and sentence descriptions of sound classes. Our goal is to obtain a classifier that capable recognizing instances classes have no available training samples, but only side information. We employ bilinear compatibility framework learn an acoustic-semantic projection between intermediate-level representations classes, i.e., acoustic embeddings. use VGGish extract deep clips, pre-trained language models (Word2Vec, GloVe, BERT) generate either label or Audio performed by linear function measures how compatible embedding are. evaluate the proposed method on small balanced dataset ESC-50 large-scale unbalanced subset AudioSet. The experimental results show performance significantly improved involving are semantically close test training. Meanwhile, demonstrate both useful for learning. Classification concatenating label/sentence generated with different models. With their hybrid concatenations, further.